Automated network-based test system for set top box devices

ABSTRACT

An automated network-based test system for set top box devices is disclosed. According to certain embodiments, the network-based testing system using Simple Network Management Protocol facilitates remote testing of thousands of set-top boxes, where groups of these set top boxes can be located in various locations that are remote from a main controller server that is running the tests remotely.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is related to concurrently filed patent applicationentitled, “An Automated Network-Based Test System For Set Top BoxDevices,” by Rajeev Tiwari and Rafael Villanueva.

TECHNICAL FIELD

The present invention is directed to a system for testing set top boxes.

BRIEF DESCRIPTION OF THE DRAWINGS

For a better understanding of the aforementioned aspects of theinvention as well as additional aspects and embodiments thereof,reference should be made to the Description of Embodiments below, inconjunction with the following drawings in which like reference numeralsrefer to corresponding parts throughout the figures.

FIG. 1 illustrates a high-level system architecture for testing set topboxes using a jump server, according to certain embodiments.

FIG. 2 illustrates a high-level system architecture for testing set topboxes using a VPN gateway and router, according to certain embodiments.

DETAILED DESCRIPTION

Methods, systems, user interfaces, and other aspects of the inventionare described. Reference will be made to certain embodiments of theinvention, examples of which are illustrated in the accompanyingdrawings. While the invention will be described in conjunction with theembodiments, it will be understood that it is not intended to limit theinvention to these particular embodiments alone. On the contrary, theinvention is intended to cover alternatives, modifications andequivalents that are within the spirit and scope of the invention. Thespecification and drawings are, accordingly, to be regarded in anillustrative rather than a restrictive sense.

Moreover, in the following description, numerous specific details areset forth to provide a thorough understanding of the present invention.However, it will be apparent to one of ordinary skill in the art thatthe invention may be practiced without these particular details. Inother instances, methods, procedures, components, and networks that arewell known to those of ordinary skill in the art are not described indetail to avoid obscuring aspects of the present invention.

Automated testing of end-user devices is described in patent applicationSer. No. 12/566,041, entitled, “Method And System For Automated Test ofEnd-User Devices,” filed Sep. 24, 2009, and is hereby incorporated byreference in its entirety.

According to certain embodiments, a network-based testing system is usedfor testing multimedia devices such as set top boxes.

According to certain embodiments, the network-based testing system cantest traditional non-IP based set-top boxes as well as IP based set-topboxes simultaneously on a given test bench.

According to certain embodiments, a network-based based testing systemusing Simple Network Management Protocol is used for testing set topboxes.

According to certain embodiments, the network-based testing system usingSimple Network Management Protocol facilitates remote testing ofthousands of set-top boxes, where groups of these set top boxes can belocated in various locations that are remote from a main controllerserver that is running the tests remotely.

According to certain embodiments, the network-based testing system usingSimple Network Management Protocol improves testing robustness. The useof Simple Network Management Protocol to retrieve diagnostic and healthstatus information directly from the set-top box that is under testobviates the need of having to rely on test output from the videoanalyzer (Vbox) and OCR software algorithms to analyze such output.

According to certain embodiments, the network-based testing system usingSimple Network Management Protocol improves reliability on pass/failcriteria evaluations without compromising test flow execution time. Forexample, in traditional non-network based testing using video captureand OCR methods to get reading values slows down test execution time.

According to certain embodiments, the network-based testing system usingSimple Network Management Protocol minimizes dependency on set-top boxmiddleware changes.

According to certain embodiments, the network-based testing system usingSimple Network Management Protocol reduces test time for pass/failcriteria evaluations. According to certain embodiments, thenetwork-based testing system using Simple Network Management Protocolreduces set-top box latency time because set-top boxes react quicker toSNMP based commands. Further, the SNMP based commands can retrievereading values in less time than that of traditional OCR based testingmethods. Further, according to certain embodiments, the development ofSNMP based testing software requires less effort and the time forrunning the test scripts are reduced as compared to traditional OCRbased testing methods.

According to certain embodiments, some of the STB relatedfunctionalities that are tested include the following non-limiting listof functionalities: a) power and boot up detection functionality, b) STBprovisioning status (e.g., channel map, user guide), c) tuner status(out-of-band and in-band tuner health status, such as frequency, signalto noise ratio, carrier lock detection, etc), d) purchases status (suchas verifying if any pending pay-per-view events generated by asubscriber are still residing in the STB and has not be cleared), e)code modules versions (for example, check to see if the code moduleversions are up to date), f) hard drive status (for example, check forread/write errors and check for bad sectors in the hard drive), g) poweroutlet (for example, check to see if power outlet is operatingproperly), h) analog video & audio outputs (for example, check to see ifComposite, Component, RF, Baseband Audio types of output are present),i) digital video & audio outputs (for example, check to see ifHigh-Definition Multimedia Interface, Sony/Philips Digital InterfaceFormat, Coaxial types of outputs are present), j) clear the content onthe hard drive and use smart drive parameters, k) USB status (check tosee if the USB port of the STB is able to power up the STB and whetherthe USB port is able to recognize a given device that is connected tothe USB port), l) restore factory settings, m) IP address, MAC address,serial number and network parameter detection functionality, n) checkSTB software and hardware version, o) video & audio quality parameters(macroblocking detection).

FIG. 1 illustrates a high-level system architecture for testing set topboxes using a jump server, according to certain embodiments. FIG. 1shows a testing system 100 that includes a virtual local area network(VLAN) 120 that, in turn, includes a plurality of video and audioanalyzers 104 a-d (herein referred to as “Vbox”), a quick test (QT)secondary controller 106 b, and a video and audio generator (QAMmodulator) 107. The plurality of video and audio analyzers 104 a-d(Vboxes) are in communication with a quick test (QT) main controller 106a via a virtual private network (VPN) gateway 119. The plurality ofvideo and audio analyzers 104 a-d (Vboxes) are part of the test bench atthe testing system 100. FIG. 1 also shows a plurality of set top boxes102 a-d. The plurality of set top boxes 102 a-d are the devices undertest and are connected to corresponding video and audio analyzers(Vboxes) 104 a-d so that tests can be conducted on the set-top boxes.According to certain embodiments, quick test (QT) main controller 106 ais remote from VLAN 120 and the STBs 102 a-d. Thus, the quick test (QT)main controller 106 a can remotely control and manage the testing ofmultiple sets of STBs that are located in multiple remote locations(only one such location is shown in FIG. 1). FIG. 1 also shows a virtualprivate network (VPN) gateway 110, a jump server 108 associated with aremote virtual local area network (VLAN) 112 of a headend office, aheadend controller 114 of the headend office, and a headend radiofrequency (RF) feed 116. FIG. 1 shows only 4 set top boxes (STBs) and 4corresponding Vboxes. However, the embodiments are not limited to only 4STBs and 4 corresponding Vboxes. The number of STBs and correspondingVboxes may vary from implementation to implementation. According tocertain embodiments, each testing system may include as many as 24 STBsand 24 corresponding Vboxes. According to certain embodiments, eachtesting system may include more than 24 STBs and 24 correspondingVboxes. According to certain embodiments, each testing system mayinclude fewer than 24 STBs and 24 corresponding Vboxes.

According to certain embodiments, each of the STBs 102 a-d is connectedto corresponding video and audio analyzer (Vbox) 104 a-d via video andaudio cables, for example. Each of the Vboxes 104 a-d in VLAN 120 isconnected via Ethernet to quick test (QT) main controller 106 a via VPNgateway 119. The quick test secondary controller 106 b is connected tovideo and audio generator (QAM modulator) 107 via a USB port. The QAMmodulator is connected (107 b) to the Vboxes 104 a-d via radio frequencycoaxial cables, for example. According to certain embodiments, jumpserver 108 communicates with quick test (QT) main controller 106 athrough the Internet via VPN gateway 110 to the VPN where the quick test(QT) main controller 106 a resides.

According to certain embodiments, quick test (QT) main controller 106 aexecutes a software application that causes a scanner to scan therespective serial number and media access control (MAC) address of eachSTB 102 a-d of the plurality of STBs under test in the testing system100. Thus, test system 100 is not dependent on an optical characterreader (OCR) to capture the serial number and MAC address of each STBunder test, nor is test system 100 dependent on OCR methods andapplications to analyze the video and audio output signals from the STBsunder test. Quick test (QT) main controller 106 a establishescommunication with jump server 108 via VPN gateway 110 using SecureShell (SSH) protocol. SSH is an encrypted protocol that allows remotelogin over an unsecured network such as the Internet. Jump server 108 isassociated with a VLAN 112. Jump server 108 authenticates and authorizesquick test (QT) main controller 106 a. After authentication andauthorization of quick test (QT) main controller 106 a, jump server 108allows quick test (QT) main controller 106 a to resolve the host IPvalues to be provisioned to the STBs 102 a-d by headend controller 114in VLAN 112 such that each of the STBs 102 a-d can communicate with thequick test (QT) main controller 106 a using Simple Network ManagementProtocol (SNMP).

According to certain embodiments, headend controller 114 provides videoand audio services to the STBs under test in testing system 100.Non-limiting examples of video and audio services provided to the STBsunder test include: 1) middleware codes for updating the STB, and 2)messages to enable channel lineup, and 3) digital video recording (DVR)services.

According to certain embodiments, a testing software application on thequick test (QT) controller 106 a executes a test script (for testing theSTBs) and sends a request to jumper server 108 asking the jump server108 to execute SNMP commands in order to collect diagnostic and healthstatus of each of the STBs under test in testing system 100. Further,such SNMP commands provides an additional channel of communicationbetween QT main controller and the STBs. Thus, QT main controller cancause the Vboxes 104 a-d to generate IR signals for sending such IRsignals to corresponding STBs 102 a-d in order to test the STBs 102 a-d.Further, the headend controller 114 can perform troubleshooting anddebugging of any problems in the STBs under test. In prior art testingsystems of STBs, the Vboxes had to rely on optical character reader(OCR) applications to analyze video and audio output signals that areoutputed from the STBs as part of the testing the STBs. Such OCR (e.g.,video OCR) applications perform analysis of the output signals (to testvarious functionalities of the STBs) relatively poorly. The use of thejump server 108 to allow quick test (QT) main controller 106 a toresolve the host IP values to be provisioned to the STBs 102 a-d byheadend controller 114 in VLAN 112 facilitates the testing oftraditional STBs as well as IP-based STBs at the same time in testsystem 100 (in other words, the testing system is designed for IP-basedSTBs but at the same time it is flexible enough to accommodatetraditional STBs at a given test bench, wherein the traditional STBs aretested using traditional OCR methods, for example). Such an SNMP testingapproach is very flexible because it is cross-platform compatible withthe latest networking technologies and operating systems. For example,the QT main controller is not constrained to be in the same VLAN as theVboxes. Thus, the quick test (QT) main controller 106 a can remotelycontrol and manage the testing of multiple sets of STBs that are locatedin multiple remote locations. Also, by this approach, the quick test(QT) main controller 106 a has an additional channel of communicationwith the STBs 102 a-d, and thus, communication between the quick test(QT) main controller 106 a and the STBs is faster and more reliable thantraditional approaches. Further, the SNMP approach is flexible to allowthe use of various software platforms for test script development suchas open source tools, as a non-limiting example. Further yet, such anapproach makes it easier for the headend controller 114 to performtroubleshooting and debugging operations remotely on the STBs 102 a-dunder test in testing system 100. For example, headend controller 114can provide radio frequency (RF) signals and DOCSIS (Data Over CableService Interface Specification) signaling to RF feed 116 via a DOCSISset-top box gateway (DSG channel) in order to test the STBs under test.

According to certain embodiments, quick test (QT) secondary 106 b incombination with QAM modulator 107, as part of the testing of the STBs,execute an application to stream video and audio test patterns to theSTBs 102 a-d in order for the respective Vboxes 104 a-d to take videoand audio output measurements from the corresponding STBs 102 a-d.Further, headend controller 114 provides Out of Band and In Band radiofrequency (RF) signals and DOCSIS (Data Over Cable Service InterfaceSpecification) signaling to RF feed 116 via Cable Modem TerminationSystem (CMTS) and a DOCSIS set-top box gateway (DSG channel) tocommunicate with the STBs under test using SNMP. Such RF signals andDOCSIS are used for testing the STBs under test. According to certainembodiments, headend controller 114 establishes the DSG channel.

According to certain embodiments, each Vbox 104 a-d captures video andaudio reading values, which are outputs from the respective STB 102 a-dunder test. Video and audio reading values include digitized raw videoand audio data (for example, the reading values include RF OUT,Composite, Component, HDMI, RF OUT, BaseBand L&R, Digital Coaxial andOptical). The respective Vboxes 104 a-d send the captured video andaudio reading values to quick test (QT) main controller 106 a. Accordingto certain embodiments, quick test (QT) main controller 106 a performsHSL (hue, saturation, lightness) color space calculations and audioparameter calculations. The HSL and audio parameter calculations areevaluated during the test flow execution.

According to certain embodiments, a sample test script for testing theSTBs is as follows:

-   -   Scan Serial and MAC bar code labels of the STB under test.    -   Obtain model identification and test list configuration of the        STB (for example, obtain list of video and audio outputs that        need to be tested for a given STB model).    -   Switch the Headend signals to RF signals as inputs to STBs (Vbox        has internal 3×1 RF switch hardware and software application        that control the RF input signals to the STBs).    -   Reboot each STB under test (for synchronization and DOCSIS-based        STBs require reboot as part of the boot up sync process).    -   Wait for reboot.    -   Open connection to Jump Server (a software application on quick        test (QT) main controller opens a port connection to jump server        using SSH protocol).    -   Scan network based on MAC bar code on each STB under test in        order to resolve IP address of each STB as decided by the        headend controller    -   Get OID (object identifier) ‘RF HOST IP’ and evaluate according        to resolved IP (Confirmation for STB IP acquisition and confirm        through OID for MAC barcode label match of the respective STB        under test).    -   Get OID ‘Firmware/middleware Version’ and evaluate if it is the        correct version as required.    -   Get OID ‘Out of Band Tuner’ and evaluate for health status (Out        Of Band Tuner handles provisioning messages and other messages        from Headend controller to STB. The parameters that are        evaluated include: Carrier Lock and signal-to-noise ratio (SNR)        status. For example, the expected values for both readings are        ‘YES’ and ‘GOOD’, respectively).    -   Get OID ‘InBand Tuner 1’ and evaluate for Corr/UnCorr Bytes (the        InBand Tuner processes video & audio and corrected bytes. The        uncorrected blocks are checked for problems on the MPEG        transport stream that is decoded in the STB. For example, the        pass criteria is: 1) Corrected Bytes counter less than 20 in a        time window of 20 seconds, and 2) Uncorrected Blocks are less        than 2 in a 20 second time window).    -   Get OID ‘InBand Tuner 2’ and evaluate for Corrected/UnCorrected        Bytes (STBs with DVR services include an additional InBand        Tuner]    -   Get OID ‘stbDvrFailures’ and evaluate for HDD health status (STB        under test provides status for hard drive recognition. Problems        related to the hard drive on DVR can be found by evaluating        embedded diagnostic information. The evaluation is based on the        status values retrieved from STB under test, and the expected        values must be either ‘true’, ‘false’ as the case may be).    -   Get OID ‘stbHDMIstate’ and evaluate for HDMI health status (for        example, evaluate the HDMI connection port status and the pass        criteria value is ‘connectedPowerOn(2)’)    -   Switch local RF Video & Audio Test pattern signals to RF input        (Vbox has internal 3×1 RF switch hardware and software        application that controls the RF input signals to the STB. The        quick test (QT) main controller controls the Vbox).    -   Tune Test Channel (the quick test (QT) main controller sends IR        (Infrared) commands to STB to tune channel.    -   Get Baseband Video HSL measurements.    -   Get BaseBand Audio Amplitude, Frequency and THD (Total Harmonic        Distortion)    -   Get Video & Audio HDMI measurements (QT Main controller gets        these measurements from STB).    -   Clear HDD (Hard Drive Disk) (the QT main controller sends IR        commands to STB to clear recordings in the HDD).    -   Close connection to Jump Server.

FIG. 2 illustrates a high-level system architecture for testing set topboxes using a router, according to certain embodiments. FIG. 2 shows atesting system 200 that includes a virtual local area network (VLAN) 220that, in turn, includes a plurality of video and audio analyzers 204 a-d(herein referred to as “Vbox”), a quick test (QT) secondary controller206 b, and a video and audio generator (QAM modulator) 207. Theplurality of video and audio analyzers 204 a-d (Vboxes) are incommunication with a quick test (QT) main controller 206 a via a virtualprivate network (VPN) gateway 219. The plurality of video and audioanalyzers 204 a-d (Vboxes) are part of the test bench at the testingsystem 200. FIG. 2 also shows a plurality of set top boxes 202 a-d. Theplurality of set top boxes 202 a-d are the devices under test and areconnected to corresponding video and audio analyzers (Vboxes) 204 a-d sothat tests can be conducted on the set-top boxes. According to certainembodiments, quick test (QT) main controller 206 a is remote from VLAN220 and the STBs 202 a-d. Thus, the quick test (QT) main controller 206a can remotely control and manage the testing of multiple sets of STBsthat are located in multiple remote locations (only one such location isshown in FIG. 2). FIG. 2 also shows a virtual private network (VPN)gateway 210, a router 208 that enables communication with a remotevirtual local area network (VLAN) 212 of a headend office, a headendcontroller 214 of the headend office, and a headend radio frequency (RF)feed 216. FIG. 2 shows only 4 set top boxes (STBs) and 4 correspondingVboxes. However, the embodiments are not limited to only 4 STBs and 4corresponding Vboxes. The number of STBs and corresponding Vboxes mayvary from implementation to implementation. According to certainembodiments, each testing system may include as many as 24 STBs and 24corresponding Vboxes. According to certain embodiments, each testingsystem may include more than 24 STBs and 24 corresponding Vboxes.According to certain embodiments, each testing system may include fewerthan 24 STBs and 24 corresponding Vboxes.

According to certain embodiments, each of the STBs 202 a-d is connectedto corresponding video and audio analyzer (Vbox) 204 a-d via video andaudio cables, for example. Each of the Vboxes 204 a-d in VLAN 220 isconnected via Ethernet to quick test (QT) main controller 206 a via VPNgateway 219. The quick test secondary controller 206 b is connected tovideo and audio generator (QAM modulator) 207 via a USB port. The QAMmodulator is connected (207 b) to the Vboxes 204 a-d via radio frequencycoaxial cables, for example. According to certain embodiments, router208 communicates with quick test (QT) main controller 206 a through theInternet via VPN gateway 210 to the VPN where the quick test (QT) maincontroller 206 a resides.

According to certain embodiments, quick test (QT) main controller 206 aexecutes a software application that causes a scanner to scan therespective serial number and media access control (MAC) address of eachSTB 202 a-d of the plurality of STBs under test in the testing system200. Thus, test system 200 is not dependent on an optical characterreader (OCR) to capture the serial number and MAC address of each STBunder test, nor is test system 200 dependent on OCR methods andapplications to analyze the video and audio output signals from the STBsunder test. Quick test (QT) main controller 206 a establishes aconnection to VLAN 212 (where the headend controller 212 resides) viarouter 208 via VPN gateway 210. Once the quick test (QT) main controller206 a establishes a connection to VLAN 212 via router 208, the quicktest (QT) main controller 206 a resolves the host IP values to beprovisioned to the STBs 202 a-d by headend controller 214 in VLAN 212such that each of the STBs 202 a-d can communicate with the quick test(QT) main controller 206 a using Simple Network Management Protocol(SNMP).

According to certain embodiments, headend controller 214 provides videoand audio services to the STBs under test in testing system 200.Non-limiting examples of video and audio services provided to the STBsunder test include: 1) middleware codes for updating the STB, and 2)messages to enable channel lineup, and 3) digital video recording (DVR)services.

According to certain embodiments, a testing software application on thequick test (QT) controller 206 a executes SNMP commands in order tocollect diagnostic and health status of each of the STBs under test intesting system 200. Further, such SNMP commands provides an additionalchannel of communication between QT main controller and the STBs. Thus,QT main controller can cause the Vboxes 204 a-d to generate IR signalsfor sending such IR signals to corresponding STBs 202 a-d in order totest the STBs 202 a-d. Further, the headend controller 214 can performtroubleshooting and debugging of any problems in the STBs under test. Inprior art testing systems of STBs, the Vboxes had to rely on opticalcharacter reader (OCR) applications to analyze video and audio outputsignals that are outputted from the STBs as part of the testing theSTBs. Such OCR (e.g., video OCR) applications perform analysis of theoutput signals (to test various functionalities of the STBs) relativelypoorly. Quick test (QT) main controller 206 a, via router 208, resolvesthe host IP values to be provisioned to the STBs 202 a-d by headendcontroller 214 in VLAN 212 and thus facilitates the testing oftraditional STBs as well as IP-based STBs at the same time in testsystem 200, as previously explained herein. Such an SNMP testingapproach is very flexible because it is cross platform compatible withthe latest networking technologies and operating systems. For example,the QT main controller is not constrained to be in the same VLAN as theVboxes. Thus, the quick test (QT) main controller 206 a can remotelycontrol and manage the testing of multiple sets of STBs that are locatedin multiple remote locations. Also, by this approach, the quick test(QT) main controller 206 a has an additional channel of communicationwith the STBs 202 a-d and thus communication between the quick test (QT)main controller 206 a and the STBs is faster and more reliable. Further,the SNMP approach is flexible to allow the use of various softwareplatforms for test script development such as open source tools, as anon-limiting example. Further yet, such an approach makes it easier forthe headend controller 214 to perform troubleshooting and debuggingoperations remotely on the STBs 202 a-d under test in testing system200. For example, headend controller 214 can provide radio frequency(RF) signals and DOCSIS (Data Over Cable Service InterfaceSpecification) signaling to RF feed 216 via a DOCSIS set-top box gateway(DSG channel) in order to test the STBs under test.

According to certain embodiments, quick test (QT) secondary 206 b incombination with QAM modulator 207, as part of the testing of the STBs,execute an application to stream video and audio test patterns to theSTBs 202 a-d in order for the respective Vboxes 204 a-d to take videoand audio output measurements from the corresponding STBs 202 a-d.Further, headend controller 214 provides Out of Band and In Band radiofrequency (RF) signals and DOCSIS (Data Over Cable Service InterfaceSpecification) signaling to RF feed 216 via a Cable Modem TerminationSystem (CMTS) and a DOCSIS set-top box gateway (DSG channel) tocommunicate with the STBs under test using SNMP. Such RF signals andDOCSIS are used for testing the STBs under test. According to certainembodiments, headend controller 214 establishes the DSG channel.

According to certain embodiments, each Vbox 204 a-d captures video andaudio reading values, which are outputs from the respective STB 202 a-dunder test. Video and audio reading values include digitized raw videoand audio data (for example, the reading values include RF OUT,Composite, Component, HDMI, RF OUT, BaseBand L&R, Digital Coaxial andOptical). The respective Vboxes 204 a-d send the captured video andaudio reading values to quick test (QT) main controller 206 a. Accordingto certain embodiments, quick test (QT) main controller 206 a performsHSL (hue, saturation, lightness) color space calculations and audioparameter calculations. The HSL and audio parameter calculations areevaluated during the test flow execution.

According to certain embodiments, a sample test script for testing theSTBs is as follows:

-   -   Scan Serial and MAC bar code labels of the STB under test.    -   Obtain model identification and test list configuration of the        STB (for example, obtain list of video and audio outputs that        need to be tested for a given STB model).    -   Switch the Headend signals to RF signals as inputs to STBs (Vbox        has internal 3×1 RF switch hardware and software application        that control the RF input signals to the STBs).    -   Reboot each STB under test (for synchronization and DOCSIS-based        STBs require reboot as part of the boot up sync process).    -   Wait for reboot.    -   Scan network based on MAC bar code on each STB under test in        order to resolve IP address of each STB as decided by the        headend controller    -   Get OID (object identifier) ‘RF HOST IP’ and evaluate according        to resolved IP (Confirmation for STB IP acquisition and confirm        through OID for MAC barcode label match of the respective STB        under test).    -   Get OID ‘Firmware/middleware Version’ and evaluate if it is the        correct version as required.    -   Get OID ‘Out of Band Tuner’ and evaluate for health status (Out        Of Band Tuner handles provisioning messages and other messages        from Headend controller to STB. The parameters that are        evaluated include: Carrier Lock and signal-to-noise ratio (SNR)        status. For example, the expected values for both readings are        ‘YES’ and ‘GOOD’, respectively).    -   Get OID ‘InBand Tuner 1’ and evaluate for Corr/UnCorr Bytes (the        InBand Tuner processes video & audio and corrected bytes. The        uncorrected blocks are checked for problems on the MPEG        transport stream that is decoded in the STB. For example, the        pass criteria is: 1) Corrected Bytes counter less than 20 in a        time window of 20 seconds, and 2) Uncorrected Blocks are less        than 2 in a 20 second time window).    -   Get OID ‘InBand Tuner 2’ and evaluate for Corrected/UnCorrected        Bytes (STBs with DVR services include an additional InBand        Tuner]    -   Get OID ‘stbDvrFailures’ and evaluate for HDD health status (STB        under test provides status for hard drive recognition. Problems        related to the hard drive on DVR can be found by evaluating        embedded diagnostic information. The evaluation is based on the        status values retrieved from STB under test, and the expected        values must be either ‘true’, ‘false’ as the case may be).    -   Get OID ‘stbHDMIstate’ and evaluate for HDMI health status (for        example, evaluate the HDMI connection port status and the pass        criteria value is ‘connectedPowerOn(2)’)    -   Switch local RF Video & Audio Test pattern signals to RF input        (Vbox has internal 3×1 RF switch hardware and software        application that controls the RF input signals to the STB. The        quick test (QT) main controller controls the Vbox).    -   Tune Test Channel (the quick test (QT) main controller sends IR        (Infrared) commands to STB to tune channel.    -   Get Baseband Video HSL measurements.    -   Get BaseBand Audio Amplitude, Frequency and THD (Total Harmonic        Distortion)    -   Get Video & Audio HDMI measurements (QT Main controller gets        these measurements from STB).    -   Clear HDD (Hard Drive Disk) (the QT main controller sends IR        commands to STB to clear recordings in the HDD).    -   Check USB status and optical connections status.    -   Check smart drive parameters to evaluate the health status of        the hard disk drive.    -   Obtain diagnostic information for Ultra High Definition (UHD)        and 4K resolutions.    -   Obtain network characteristics for security and data analysis.

In the foregoing specification, embodiments of the invention have beendescribed with reference to numerous specific details that may vary fromimplementation to implementation. The specification and drawings are,accordingly, to be regarded in an illustrative rather than a restrictivesense.

We claim:
 1. A system for testing a plurality of set-top boxes undertest, the system comprising: a VLAN of a plurality of VLANs, the VLANcomprising a plurality of video and audio analyzers wherein each videoand audio analyzer of at least a subset of the plurality of video andaudio analyzers sends input signals to a corresponding set-top box ofthe plurality of set-top boxes under test for testing the correspondingset-top box; a main test controller that is remote from the VLAN,wherein the main test controller communicates with the plurality ofvideo and audio analyzers via Ethernet; a router that communicates witha headend controller that is residing at the headend office; and aheadend radio frequency feed that is in communication with each videoand audio analyzer of at least a subset of the plurality of video andaudio analyzers.
 2. The system of claim 1, further comprises a secondarytest controller that controls a video and audio generator that streamsvideo and audio test patterns to at least a subset of the plurality ofset-top boxes under test via corresponding video and audio analyzers. 3.The system of claim 1, wherein the main test controller analyzes outputsignals outputted by at least a subset of the plurality of set-top boxesunder test.
 4. The system of claim 1, wherein the main test controlleruses the router to communicate with a remote headend controller toresolve host IP values for each set-top box of at least a subset of theplurality of set-top boxes under test.
 5. The system of claim 1, whereinthe main test controller causes each video and audio analyzer of atleast a subset of the plurality of video and audio analyzers to generateinfra red signals for sending the infra red signals to a correspondingset-top box under test.